# Document Understanding

DeepSeek-VL2 is a series of advanced large-scale mixture of experts (MoE) visual language models, significantly improved compared to its predecessor DeepSeek-VL. This model series demonstrates exceptional capabilities across various tasks, including visual question answering, optical character recognition, document/table/chart understanding, and visual localization. Comprising three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small, and DeepSeek-VL2, with 1 billion, 2.8 billion, and 4.5 billion active parameters respectively, DeepSeek-VL2 achieves competitive or state-of-the-art performance against existing dense and MoE-based open-source models, even with a similar or fewer number of active parameters.

mPLUG-DocOwl 1.5 is a unified structural learning model dedicated to OCR-free document understanding, achieving direct comprehension of documents through deep learning technologies without the need for traditional Optical Character Recognition (OCR). The model can handle various types of images, including documents, web pages, tables, and charts, supporting structural-aware document parsing, multi-granularity text recognition and localization, as well as question-and-answer capabilities. The development of mPLUG-DocOwl 1.5 is driven by the demand for automated and intelligent document understanding, aiming to enhance the efficiency and accuracy of document processing. Its open-source nature also facilitates further research and application in both academia and industry.

EAGLE is a series of high-resolution, vision-centric multimodal large language models (LLMs) designed to enhance the perception capabilities of multimodal LLMs through a combination of visual encoders and varied input resolutions. The model features a 'CLIP+X' fusion based on channel connections, suitable for visual experts trained on different architectures (ViT/ConvNets) and domains (detection/segmentation/OCR/SSL). The EAGLE model family supports input resolutions over 1K and excels in multimodal LLM benchmarks, particularly in resolution-sensitive tasks such as optical character recognition and document understanding.

Adobe Acrobat's AI assistant is an innovative feature integrated within Acrobat. It empowers users to understand document content through question-based interaction, swiftly retrieve information, and generate content. This functionality is particularly beneficial for individuals who need to extract key information from documents and quickly create emails, presentations, meeting minutes, and more. The AI assistant can generate document summaries, offer suggested questions based on document content, and help users explore and understand information more efficiently. Furthermore, Adobe Acrobat's AI assistant adheres to Adobe's AI ethics principles, ensuring user data security and transparency.

mPLUG-DocOwl is a modular multimodal large language model designed for document understanding, which can process OCR-free document understanding tasks. The model showcases exceptional performance, supporting various tasks including document visual Q&A, information Q&A, and chart Q&A. Users can experience its powerful functionalities through the online demonstration provided by the model.

DocLLM is a platform that provides a multimodal document understanding model, aiming to process both text and spatial layout within enterprise documents, delivering performance superior to existing large language models (LLMs). Its model employs a lightweight extension, avoiding expensive image encoders and focusing on bounding box information to incorporate spatial layout structure. By deconstructing the attention mechanism in classic Transformers, it captures cross-modality alignment between text and spatial modalities. Furthermore, a pre-training objective is designed to learn text paragraph filling, addressing the irregular layouts and heterogeneous content frequently encountered in visual documents. This solution outperforms existing LLMs on 14 tasks across 16 datasets and demonstrates good generalization capabilities on 5 previously unseen datasets.